Researchers from Columbia and Harvard have successfully used AI tools to engineer a portion of the E. coli ribosome that functions without isoleucine, one of the 20 standard amino acids. By replacing isoleucine with similar amino acids such as valine through iterative testing and deep-learning software, the team created an isoleucine-free small subunit in the bacteria. Although these engineered cells survived, they experienced slower growth rates than unmodified strains. This research investigates the possibility of simplified genetic codes and offers clues about how early life may have operated with fewer amino acids.
This research presents a scalable method for extracting linear representations of concepts within large-scale AI models, including language, vision-language, and reasoning models. By mapping these internal representations, the authors demonstrate how to steer model behavior to mitigate misalignment, expose vulnerabilities, and enhance capabilities beyond traditional prompting. The study also shows that these concept representations are transferable across languages and can be combined for multi-concept steering. Additionally, the approach provides a superior method for monitoring misaligned content like hallucinations and toxicity compared to direct output judgment models.
Key points:
- Scalable extraction of linear concept representations
- Model steering for safety and capability enhancement
- Cross-language transferability and multi-concept steering
- Monitoring of hallucinations and toxic content via internal states
Key points:
- Scalable extraction of linear concept representations
- Model steering for safety and capability enhancement
- Cross-language transferability and multi-concept steering
- Monitoring of hallucinations and toxic content via internal states
Researchers have developed a new method for identifying concept representations within neural networks, offering a way to monitor and control artificial intelligence from the inside. By locating specific numeric patterns that represent concepts like truthfulness, this approach allows for more effective steering of model behavior compared to existing methods.
Key points include:
- Identification of internal numeric patterns representing high-level concepts.
- Improved performance in controlling AI responses during coding tasks.
- Potential for automated monitoring of factual correctness without human intervention.
Key points include:
- Identification of internal numeric patterns representing high-level concepts.
- Improved performance in controlling AI responses during coding tasks.
- Potential for automated monitoring of factual correctness without human intervention.
A specialized implementation of a 25,000-parameter decoder-only transformer designed to run on an unmodified Commodore 64. Written in hand-coded 6502 assembly, the model features real multi-head causal self-attention, RMSNorm, and softmax, achieving functionality similar to modern LLM architectures despite the extreme hardware constraints of a 1 MHz processor.
Key technical details include:
- Uses int8 quantized parameters with per-tensor shift scaling.
- Implements fixed-point arithmetic (Q8.8) for activations.
- Features a 128-token BPE vocabulary and a 20-token context window.
- Includes tools for quantization-aware training (QAT) to ensure model accuracy on integer hardware.
- Capable of running on real C64 hardware or emulators like VICE, with performance averaging 60 seconds per token.
Key technical details include:
- Uses int8 quantized parameters with per-tensor shift scaling.
- Implements fixed-point arithmetic (Q8.8) for activations.
- Features a 128-token BPE vocabulary and a 20-token context window.
- Includes tools for quantization-aware training (QAT) to ensure model accuracy on integer hardware.
- Capable of running on real C64 hardware or emulators like VICE, with performance averaging 60 seconds per token.
This article explores the growing trend of using small language models (SLMs) to power autonomous AI agents locally on consumer hardware. It discusses how recent advancements in model efficiency allow these smaller, specialized models to perform complex reasoning and tool-use tasks previously reserved for much larger models. The guide covers the benefits of local deployment, such as privacy, reduced latency, and cost savings, while outlining technical strategies for implementing agentic workflows using frameworks like LangChain or AutoGPT with quantized SLMs.
This article introduces Scikit-LLM, a Python library that integrates large language models like OpenAI's GPT with the Scikit-learn framework to simplify text analysis tasks. It explains and demonstrates two primary classification methods: zero-shot classification, which assigns labels based solely on the model's general knowledge without prior examples, and few-shot classification, which uses a small set of labeled examples within the prompt to improve accuracy. By following a Scikit-learn-style workflow using fit() and predict() methods, users can easily implement these advanced NLP techniques for tasks such as sentiment analysis and topic labeling.
This article demonstrates how to perform text summarization using the scikit-llm library, which provides a simple interface for utilizing large language models within a scikit-learn style workflow. The guide walks through installing the necessary dependencies and implementing both extractive and abstractive summarization techniques on sample text data.
Key topics include:
- Introduction to the scikit-llm library
- Implementing abstractive summarization using LLMs
- Using scikit-llm for text classification and clustering tasks
- Practical code examples for integrating LLM capabilities into machine learning pipelines
Key topics include:
- Introduction to the scikit-llm library
- Implementing abstractive summarization using LLMs
- Using scikit-llm for text classification and clustering tasks
- Practical code examples for integrating LLM capabilities into machine learning pipelines
OpenMythos is an open-source PyTorch project by Kye Gomez that proposes a theoretical reconstruction of Anthropic's Claude Mythos architecture. Instead of standard transformer layers, it suggests a Recurrent-Depth Transformer (RDT) design where weights loop through multiple iterations to increase reasoning depth during inference. By combining Mixture-of-Experts with Multi-Latent Attention and stability constraints, the model achieves performance parity between 770M parameters and a 1.3B parameter standard transformer.
* open-source PyTorch reconstruction of claude mythos
* proposes recurrent-depth transformer architecture
* reasoning depth scales via inference-time loops rather than parameter count
* uses mixture-of-experts for domain breadth
* implements multi-latent attention to reduce memory usage
* employs lti injection and adaptive computation time for stability
* achieves 1.3b parameter performance with only 770m parameters
* open-source PyTorch reconstruction of claude mythos
* proposes recurrent-depth transformer architecture
* reasoning depth scales via inference-time loops rather than parameter count
* uses mixture-of-experts for domain breadth
* implements multi-latent attention to reduce memory usage
* employs lti injection and adaptive computation time for stability
* achieves 1.3b parameter performance with only 770m parameters
Mounia Lalmas, a Senior Director of Research and Head of Tech Research at Spotify, has been appointed an honorary doctorate at the University of Gothenburg's Faculty of Science and Technology. An expert in engagement and recommendation systems, Lalmas discusses her work bridging the gap between academic research and large-scale industrial application.
These working notes by Russ Tedrake cover nonlinear dynamics and control with a specific focus on mechanical systems. The material explores how to achieve robust, efficient, and graceful robot movement through the integration of mechanical design, passive dynamics, and nonlinear control synthesis. Rather than relying solely on model-free approaches, the text emphasizes using the underlying structure of dynamical equations to develop more data-efficient and robust algorithms via optimization and machine learning.
Main topics include:
* Model systems such as pendulums, acrobots, cart-poles, and quadrotors
* Simple models of walking and running dynamics
* Nonlinear planning and control using trajectory optimization and LQR
* Lyapunov analysis for stability and reachability
* Estimation techniques including Kalman filters and Bayesian methods
* Learning-based approaches such as imitation learning, policy search, and system identification
* Contact-implicit trajectory optimization and hybrid systems
Main topics include:
* Model systems such as pendulums, acrobots, cart-poles, and quadrotors
* Simple models of walking and running dynamics
* Nonlinear planning and control using trajectory optimization and LQR
* Lyapunov analysis for stability and reachability
* Estimation techniques including Kalman filters and Bayesian methods
* Learning-based approaches such as imitation learning, policy search, and system identification
* Contact-implicit trajectory optimization and hybrid systems
